feat: ✨ Official docker images for docTR #1322

odulcy-mindee · 2023-09-18T16:23:50Z

TODO

TF Dockerfile
Still heavy TBH:

REPOSITORY                                 TAG                                              IMAGE ID       CREATED             SIZE
test_doctr_gpu                             latest                                           936c9e203093   17 minutes ago      8.22GB

PT Dockerfile
CI job to publish docTR image

codecov · 2023-09-18T16:40:24Z

Codecov Report

Merging #1322 (9f7c649) into main (3271460) will increase coverage by 0.01%.
Report is 3 commits behind head on main.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1322      +/-   ##
==========================================
+ Coverage   95.78%   95.79%   +0.01%     
==========================================
  Files         154      154              
  Lines        6902     6902              
==========================================
+ Hits         6611     6612       +1     
+ Misses        291      290       -1

Flag	Coverage Δ
unittests	`95.79% <100.00%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
doctr/models/classification/predictor/pytorch.py	`95.45% <100.00%> (ø)`
doctr/models/detection/predictor/pytorch.py	`95.23% <100.00%> (ø)`
doctr/models/kie_predictor/pytorch.py	`100.00% <100.00%> (ø)`
doctr/models/predictor/pytorch.py	`100.00% <100.00%> (ø)`
doctr/models/recognition/parseq/pytorch.py	`97.16% <100.00%> (ø)`
doctr/models/recognition/parseq/tensorflow.py	`97.33% <100.00%> (ø)`
doctr/models/recognition/predictor/pytorch.py	`91.66% <100.00%> (ø)`
doctr/models/recognition/vitstr/pytorch.py	`100.00% <100.00%> (ø)`
doctr/models/recognition/vitstr/tensorflow.py	`97.61% <100.00%> (ø)`

... and 1 file with indirect coverage changes

.github/workflows/docker.yml

felixT2K · 2023-09-26T10:53:58Z

Further todo's:

Add a documentation page
Add a badge in the readme which links to the published images
Rethink the CI job (push to hub on publish - as discussed in Slack)

@odulcy-mindee Anything missing ? ^^

Dockerfile

odulcy-mindee · 2023-09-26T14:55:13Z

Further todo's:

That's good ! 👍

This reverts commit 51dd73d.

README.md

.github/workflows/public_docker_images.yml

odulcy-mindee · 2023-09-27T16:39:42Z

@felixdittrich92 FYI:

test_doctr                                 latest                                           523853251752   About a minute ago   4.01GB
test_doctr_gpu                             latest                                           a7866d727078   9 minutes ago        9.02GB

felixT2K

@odulcy-mindee Really good job that's a great and useful feature 👍🏼

I added only a few comments and questions

.github/workflows/public_docker_images.yml

README.md

felixT2K · 2023-09-28T12:03:00Z

.github/workflows/public_docker_images.yml

+
+      - name: Push Docker image
+        # Push only if the CI is not triggered by "PR on main"
+        if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'


I am not 100% sure that this works on publish (release)

@felixT2K Mmmh, actually it works for tags but it's not done on publish: [released] as I'm afraid the CI job metadata from docker does not handle this event. I'll double check

@felixT2K I did a test on my fork. When a draft release is created, a new tag is not created. When the release is published, the tag is created, so it triggers the on: push: tags event 👍

odulcy-mindee · 2023-09-28T13:28:09Z

.github/workflows/public_docker_images.yml

+  pull_request:
+    branches: main
+  schedule:
+    - cron: '0 2 29 * *'  # At 02:00 on day-of-month 29


@felixT2K I changed the cron so it should be triggered tomorrow morning. We'll see if it works or not

alright 👍🏼

.github/workflows/docker.yml

felixT2K · 2023-09-28T14:16:24Z

great job 🤗 thanks

odulcy-mindee · 2023-09-28T14:39:43Z

🚀

ffalkenberg · 2023-09-29T09:08:11Z

where are the prebuilt docker images hosted?

ffalkenberg · 2023-09-29T09:13:43Z

Having reviewed your Dockerfile, it seems that it's configured to operate as root by default. This is a significant security risk and is likely unacceptable for most production environments.

odulcy-mindee · 2023-09-29T12:17:46Z

Hello @ffalkenberg,

where are the prebuilt docker images hosted?

You can find the docker images here. You can also have a look at this section in the README.md

it seems that it's configured to operate as root by default.

Yes, the container runs as root, as it did before. I understand your concern. For that, I suggest you take a look at what Jupyter has done, they have such an interesting approach to handling that, but it's really not trivial. This deserves a dedicated Pull Request to handle these user/permission issues.

odulcy-mindee · 2023-09-29T17:02:13Z

@ffalkenberg

sh: 14: python: not found

It's python3

$ /opt/python

It's a folder.

Moreover, this works:

$ docker run -it --rm -u 1000 ghcr.io/mindee/doctr:torch-py3.9.18-gpu-2023-09 bash
I have no name!@451b261aabdb:/$ python3
Python 3.10.13 (main, Sep 27 2023, 16:35:15) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
I have no name!@451b261aabdb:/$

So it's not a permission issue.

For the demo app, you can still craft your own Docker image, based on the previous modification:

FROM ghcr.io/mindee/doctr:torch-py3.9.18-gpu-2023-09

ENV DOCTR_CACHE_DIR=/app/.cache 
WORKDIR /app

COPY . .

RUN chmod -R a+w /app

You can also add a dedicated user in the container but it's out of scope of this PR.

odulcy-mindee changed the title ~~feat: ✨ tf Dockerfile~~ feat: ✨ Official docker images for docTR Sep 18, 2023

odulcy-mindee mentioned this pull request Sep 18, 2023

Update Dockerfile (GPU Support, Workdir, Permissions) #1313

Merged

odulcy-mindee marked this pull request as ready for review September 25, 2023 14:28

odulcy-mindee added type: enhancement Improvement topic: docker Docker-related os: linux Involving Linux users type: new feature New feature labels Sep 25, 2023

odulcy-mindee force-pushed the official_docker_images branch from 748a82b to dc86d0d Compare September 25, 2023 15:40

odulcy-mindee commented Sep 26, 2023

View reviewed changes

.github/workflows/docker.yml Show resolved Hide resolved

felixT2K reviewed Sep 26, 2023

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

odulcy-mindee added 12 commits September 27, 2023 15:13

feat: ✨ tf Dockerfile

80285a3

Generic Docker Image + CI

980aeb4

fix

f36a626

python3

c8905a5

Parametrize framework

e977333

Remove DOCTR_CACHE_DIR (already set by the code)

377fe8a

build-args

45ac49d

set tags for docker image

b2f2f6b

Use pip+git install

bc0e542

fix

2389e68

fix

5f2b7ed

REVERT ME

51dd73d

odulcy-mindee force-pushed the official_docker_images branch from 7dcd8e9 to 51dd73d Compare September 27, 2023 13:13

odulcy-mindee added 4 commits September 27, 2023 15:15

Revert "REVERT ME"

491e34e

This reverts commit 51dd73d.

Fix

c832a84

Add git

ee08bbd

Modify frequency

9553d16

odulcy-mindee added 2 commits September 27, 2023 16:14

Update README.md

8537524

Add Docker Badge

437167b

felixT2K reviewed Sep 27, 2023

View reviewed changes

README.md Show resolved Hide resolved

felixT2K reviewed Sep 27, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

odulcy-mindee commented Sep 27, 2023

View reviewed changes

.github/workflows/public_docker_images.yml Outdated Show resolved Hide resolved

.github/workflows/public_docker_images.yml Outdated Show resolved Hide resolved

Add support for cpu/gpu + check if doctr can be imported

947144a

odulcy-mindee added 4 commits September 27, 2023 18:40

Update Dockerfile

0ffaf7a

Few improvements

12aa2ce

Missing lib + monthly

162748c

Fix dep issue

43e7cc5

odulcy-mindee requested a review from felixT2K September 28, 2023 11:45

felixT2K suggested changes Sep 28, 2023

View reviewed changes

Address last pr comments

2849b6c

odulcy-mindee commented Sep 28, 2023

View reviewed changes

felixT2K reviewed Sep 28, 2023

View reviewed changes

.github/workflows/docker.yml Outdated Show resolved Hide resolved

cpu flag for CI + remove latest tag + update README.md

9f7c649

felixT2K approved these changes Sep 28, 2023

View reviewed changes

felixdittrich92 approved these changes Sep 28, 2023

View reviewed changes

felixdittrich92 added this to the 0.7.1 milestone Sep 28, 2023

felixdittrich92 linked an issue Sep 28, 2023 that may be closed by this pull request

Documentation for running on docker #1113

Closed

odulcy-mindee merged commit 69f6705 into mindee:main Sep 28, 2023
69 of 70 checks passed

odulcy-mindee deleted the official_docker_images branch September 28, 2023 15:52

nh2 mentioned this pull request Nov 6, 2023

Relax Pillow and OpenCV version bounds. #1373

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: ✨ Official docker images for docTR #1322

feat: ✨ Official docker images for docTR #1322

odulcy-mindee commented Sep 18, 2023 •

edited

Loading

codecov bot commented Sep 18, 2023 •

edited

Loading

felixT2K commented Sep 26, 2023 •

edited by odulcy-mindee

Loading

odulcy-mindee commented Sep 26, 2023

odulcy-mindee commented Sep 27, 2023

felixT2K left a comment •

edited

Loading

felixT2K Sep 28, 2023

odulcy-mindee Sep 28, 2023

odulcy-mindee Sep 28, 2023

felixT2K Sep 28, 2023

odulcy-mindee Sep 28, 2023

felixT2K Sep 28, 2023

felixT2K commented Sep 28, 2023

odulcy-mindee commented Sep 28, 2023

ffalkenberg commented Sep 29, 2023

ffalkenberg commented Sep 29, 2023

odulcy-mindee commented Sep 29, 2023

odulcy-mindee commented Sep 29, 2023

feat: ✨ Official docker images for docTR #1322

feat: ✨ Official docker images for docTR #1322

Conversation

odulcy-mindee commented Sep 18, 2023 • edited Loading

TODO

codecov bot commented Sep 18, 2023 • edited Loading

Codecov Report

felixT2K commented Sep 26, 2023 • edited by odulcy-mindee Loading

odulcy-mindee commented Sep 26, 2023

odulcy-mindee commented Sep 27, 2023

felixT2K left a comment • edited Loading

Choose a reason for hiding this comment

felixT2K Sep 28, 2023

Choose a reason for hiding this comment

odulcy-mindee Sep 28, 2023

Choose a reason for hiding this comment

odulcy-mindee Sep 28, 2023

Choose a reason for hiding this comment

felixT2K Sep 28, 2023

Choose a reason for hiding this comment

odulcy-mindee Sep 28, 2023

Choose a reason for hiding this comment

felixT2K Sep 28, 2023

Choose a reason for hiding this comment

felixT2K commented Sep 28, 2023

odulcy-mindee commented Sep 28, 2023

ffalkenberg commented Sep 29, 2023

ffalkenberg commented Sep 29, 2023

odulcy-mindee commented Sep 29, 2023

odulcy-mindee commented Sep 29, 2023

odulcy-mindee commented Sep 18, 2023 •

edited

Loading

codecov bot commented Sep 18, 2023 •

edited

Loading

felixT2K commented Sep 26, 2023 •

edited by odulcy-mindee

Loading

felixT2K left a comment •

edited

Loading